test discovery 2.0 by eleanorjboyd · Pull Request #25760 · microsoft/vscode-python

eleanorjboyd · 2026-02-03T17:14:24Z

Summary

This PR implements project-based test discovery for pytest, enabling multi-project workspace support. When the Python Environments API is available, the extension now discovers Python projects within workspaces and creates separate test tree roots for each project with its own Python environment.

What's New

Project-Based Testing Architecture

TestProjectRegistry: Manages the lifecycle of Python test projects, including:
- Discovering projects via Python Environments API
- Creating ProjectAdapter instances per workspace
- Computing nested project relationships for ignore lists
- Fallback to "legacy" single-adapter mode when API unavailable
ProjectAdapter: Interface representing a single Python project with test infrastructure:
- Project identity (ID, name, URI)
- Python environment from the environments API
- Test framework adapters (discovery/execution)
- Nested project ignore paths

Key Features

✅ Multi-project workspaces: Each Python project gets its own test tree root
✅ Nested project handling: Parent projects automatically ignore nested child projects via --ignore flags
✅ Graceful fallback: Falls back to legacy single-adapter mode if Python Environments API is unavailable
✅ Project root path: Python-side get_test_root_path() function returns appropriate root for test tree

Code Improvements

Standardized logging prefixes to [test-by-project] across all files
Centralized adapter creation via createTestAdapters() helper method
Extracted reusable methods for discovery, execution, and file watching

Scope & Limitations

⚠️ Important: unittest is NOT supported in this PR

This PR focuses exclusively on pytest. unittest support for project-based testing will be implemented in a future PR.

Testing

Added unit tests for TestProjectRegistry class
Added unit tests for Python-side get_test_root_path() function
Manual testing with multi-project workspaces

Related Issues

first step in: microsoft/vscode-python-environments#987

- Created ProjectAdapter and WorkspaceDiscoveryState interfaces - Added project utility functions (ID generation, scoping, nested project detection) - Updated PythonResultResolver to support optional projectId parameter - Modified populateTestTree to create project-scoped test IDs - Updated TestDiscoveryHandler to handle project-scoped error nodes Co-authored-by: eleanorjboyd <26030610+eleanorjboyd@users.noreply.github.com>

- Added project-based state maps (workspaceProjects, vsIdToProject, fileUriToProject, projectToVsIds) - Implemented discoverWorkspaceProjects() to query Python Environment API - Created createProjectAdapter() to build ProjectAdapter from PythonProject - Added createDefaultProject() for backward compatibility - Imported necessary types from environment API - Added flag to enable/disable project-based testing Co-authored-by: eleanorjboyd <26030610+eleanorjboyd@users.noreply.github.com>

- Modified activate() to check useProjectBasedTesting flag - Calls discoverWorkspaceProjects() for each workspace when enabled - Populates workspaceProjects map with discovered projects - Created activateLegacyWorkspace() for backward compatibility - Falls back to legacy mode if project discovery fails - Maintains full backward compatibility with flag disabled Co-authored-by: eleanorjboyd <26030610+eleanorjboyd@users.noreply.github.com>

Copilot

Pull request overview

This PR implements project-based test discovery for pytest, enabling multi-project workspace support where each Python project gets its own test tree root with its own Python environment. When the Python Environments API is available, the extension now discovers Python projects within workspaces and creates separate test adapters for each project.

Changes:

Introduces TestProjectRegistry for managing project lifecycle, discovery, and nested project handling
Adds ProjectAdapter interface representing individual Python projects with test infrastructure
Implements Python-side get_test_root_path() function to support project-scoped test trees
Adds comprehensive unit tests for project registry, utilities, and Python-side functions

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/client/testing/testController/controller.ts	Refactored activation to support project-based and legacy modes, added project discovery orchestration
src/client/testing/testController/common/testProjectRegistry.ts	New registry class managing Python project discovery, adapter creation, and nested project ignore lists
src/client/testing/testController/common/projectAdapter.ts	New interface defining structure for project-based test infrastructure
src/client/testing/testController/common/projectUtils.ts	Utility functions for project ID generation, display names, and adapter creation
src/client/testing/testController/common/resultResolver.ts	Added optional projectId parameter for scoping test IDs to projects
src/client/testing/testController/common/utils.ts	Modified populateTestTree to support project-scoped test IDs with separator
src/client/testing/testController/common/types.ts	Extracted ITestItemMappings interface, added optional project parameter to ITestDiscoveryAdapter
src/client/testing/testController/common/testDiscoveryHandler.ts	Updated error node creation to support project-scoped IDs
src/client/testing/testController/pytest/pytestDiscoveryAdapter.ts	Added PROJECT_ROOT_PATH env var and --ignore flags for nested projects
python_files/vscode_pytest/init.py	Implemented get_test_root_path() function to return project root or cwd for test tree structure
src/test/testing/testController/controller.unit.test.ts	Added tests for controller project-based testing functionality
src/test/testing/testController/common/testProjectRegistry.unit.test.ts	Comprehensive tests for TestProjectRegistry class
src/test/testing/testController/common/projectUtils.unit.test.ts	Tests for project utility functions (getProjectId, parseVsId, createProjectDisplayName)
python_files/tests/pytestadapter/test_discovery.py	Added test_project_root_path_env_var() to verify PROJECT_ROOT_PATH behavior
python_files/tests/pytestadapter/expected_discovery_test_output.py	Added expected output for PROJECT_ROOT_PATH test scenario
.github/instructions/testing_feature_area.instructions.md	Documented project-based testing architecture and workflow
.github/instructions/testing-workflow.instructions.md	Added learning about stubbing after refactoring

src/client/testing/testController/controller.ts

src/client/testing/testController/common/testProjectRegistry.ts

.github/instructions/testing_feature_area.instructions.md

src/client/testing/testController/common/projectUtils.ts

eleanorjboyd · 2026-02-06T04:44:23Z

python_files/tests/pytestadapter/expected_discovery_test_output.py

+# =====================================================================================
+# PROJECT_ROOT_PATH environment variable tests
+# These test the project-based testing feature where PROJECT_ROOT_PATH changes
+# the test tree root from cwd to the specified project path.


in the diagram- outline where the projects are configured in this setup

eleanorjboyd · 2026-02-06T04:46:05Z

python_files/vscode_pytest/__init__.py

            raise VSCodePytestError(
                f"Error occurred while calculating symlink equivalent from node path: {e}"
-                f"\n SYMLINK_PATH: {SYMLINK_PATH}, \n node path: {node_path}, \n cwd: {_CACHED_CWD if _CACHED_CWD else pathlib.Path.cwd()}"
+                f"\n SYMLINK_PATH: {SYMLINK_PATH}, \n node path: {node_path}, \n cwd: {_CACHED_CWD or pathlib.Path.cwd()}"


can we add a test which verifies symlinks work with the project based setup? Look at old symlink tests for reference

eleanorjboyd · 2026-02-06T04:47:22Z

src/client/testing/testController/common/projectUtils.ts

+
+/**
+ * Creates test adapters (discovery and execution) for a given test provider.
+ * Centralizes adapter creation to avoid code duplication across Controller and TestProjectRegistry.


remove line: Centralizes adapter creation to avoid code duplication across Controller and TestProjectRegistry.

eleanorjboyd · 2026-02-06T04:48:01Z

src/client/testing/testController/common/resultResolver.ts


-    constructor(testController: TestController, testProvider: TestProvider, private workspaceUri: Uri) {
+    /**
+     * Optional project ID for scoping test IDs.


in another file the export const PROJECT_ID_SEPARATOR = '@@vsc@@'; should these not be the same?

eleanorjboyd · 2026-02-06T04:49:30Z

src/client/testing/testController/common/testProjectRegistry.ts

+export class TestProjectRegistry {
+    /**
+     * Map of workspace URI -> Map of project ID -> ProjectAdapter
+     * Project IDs match Python Environments extension's Map<string, PythonProject> keys


this is confusing- clarify this concept better

eleanorjboyd · 2026-02-06T04:52:44Z

src/client/testing/testController/common/testProjectRegistry.ts

+            throw new Error(`No Python environment found for project ${projectId}`);
+        }
+
+        // Create test infrastructure


clarify test infra per project

eleanorjboyd · 2026-02-06T04:53:20Z

src/client/testing/testController/common/testProjectRegistry.ts

+    /**
+     * Identifies nested projects and returns ignore paths for parent projects.
+     */
+    private computeNestedProjectIgnores(workspaceUri: Uri): Map<string, string[]> {


add time complexity to doc string

eleanorjboyd · 2026-02-06T04:54:42Z

src/client/testing/testController/controller.ts

+     * When true, discovers and manages tests per-project using Python Environments API.
+     * When false, uses legacy single-workspace mode.
+     */
+    private readonly useProjectBasedTesting = true;


remove this feature flag- instead it will operate based on the useEnvExtension variable

eleanorjboyd · 2026-02-06T04:56:07Z

src/client/testing/testController/controller.ts

+                project.projectUri,
+                this.pythonExecFactory,
+                this.refreshCancellation.token,
+                undefined, // Interpreter not needed; adapter uses Python Environments API


update the discover test definition to specify this is how it will work (for clarity)

eleanorjboyd · 2026-02-06T04:57:13Z

src/test/testing/testController/controller.unit.test.ts

+// NOTE:
+// `PythonTestController` calls `vscode.tests.createTestController(...)` in its constructor.
+// In unit tests, `vscode` is a mocked module (see `src/test/vscode-mock.ts`) and it does not


it seems easier to maybe just add test namespace to the vscode mock module right?

## Summary This PR implements **project-based test discovery** for pytest, enabling multi-project workspace support. When the Python Environments API is available, the extension now discovers Python projects within workspaces and creates separate test tree roots for each project with its own Python environment. ## What's New ### Project-Based Testing Architecture - **TestProjectRegistry**: Manages the lifecycle of Python test projects, including: - Discovering projects via Python Environments API - Creating ProjectAdapter instances per workspace - Computing nested project relationships for ignore lists - Fallback to "legacy" single-adapter mode when API unavailable - **ProjectAdapter**: Interface representing a single Python project with test infrastructure: - Project identity (ID, name, URI) - Python environment from the environments API - Test framework adapters (discovery/execution) - Nested project ignore paths ### Key Features - ✅ **Multi-project workspaces**: Each Python project gets its own test tree root - ✅ **Nested project handling**: Parent projects automatically ignore nested child projects via `--ignore` flags - ✅ **Graceful fallback**: Falls back to legacy single-adapter mode if Python Environments API is unavailable - ✅ **Project root path**: Python-side `get_test_root_path()` function returns appropriate root for test tree ### Code Improvements - Standardized logging prefixes to `[test-by-project]` across all files - Centralized adapter creation via `createTestAdapters()` helper method - Extracted reusable methods for discovery, execution, and file watching ## Scope & Limitations > **⚠️ Important: unittest is NOT supported in this PR** > > This PR focuses exclusively on **pytest**. unittest support for project-based testing will be implemented in a future PR. ## Testing - Added unit tests for `TestProjectRegistry` class - Added unit tests for Python-side `get_test_root_path()` function - Manual testing with multi-project workspaces ## Related Issues first step in: microsoft/vscode-python-environments#987 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

eleanorjboyd and others added 18 commits January 8, 2026 11:02

Checkpoint from VS Code for cloud agent session

ad3ae47

updates

8e13f1d

formatting

133d97e

remove doc on design

aba0183

remove unneeded

61745de

adding tests for helpers

3b7cbf9

testing and refinement

cf2e75c

tests for controller

28b34dc

separators and update api calls

7b81f07

checkpoint- project test nodes

b2a3a8e

second checkpoint- ignore implemented

45675a4

cleanup cleanup everybody everywhere

267007b

remove comments

2abfbbe

refinement

4e7a325

remove unittest refs

225ff12

eleanorjboyd self-assigned this Feb 3, 2026

eleanorjboyd added feature-request Request for new features or functionality area-testing labels Feb 3, 2026

eleanorjboyd added 3 commits February 3, 2026 13:38

cleanup

29533cf

pytest tests

ca14068

test fixes

34965e3

eleanorjboyd requested a review from Copilot February 3, 2026 22:01

Copilot started reviewing on behalf of eleanorjboyd February 3, 2026 22:01 View session

Copilot AI reviewed Feb 3, 2026

View reviewed changes

eleanorjboyd added 3 commits February 3, 2026 14:10

fix

7c3c879

address comments

42cd011

fixes

145ccc8

testing logging

ef32ac2

eleanorjboyd closed this Feb 5, 2026

eleanorjboyd reopened this Feb 5, 2026

eleanorjboyd added 2 commits February 5, 2026 12:05

test fix

b4563cd

remove unneeded edits

d472e4b

eleanorjboyd commented Feb 6, 2026

View reviewed changes

eleanorjboyd changed the base branch from main to test-project-support February 6, 2026 05:01

eleanorjboyd added 2 commits February 5, 2026 21:19

lots of fun fixes

0718994

fix

9050b23

eleanorjboyd marked this pull request as ready for review February 6, 2026 05:36

eleanorjboyd merged commit 5cc9aac into microsoft:test-project-support Feb 6, 2026
44 checks passed

wdawson mentioned this pull request Feb 6, 2026

pytest discovery does not use specified environment #25718

Open

Conversation

eleanorjboyd commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's New

Project-Based Testing Architecture

Key Features

Code Improvements

Scope & Limitations

Testing

Related Issues

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

eleanorjboyd commented Feb 3, 2026 •

edited

Loading